The invention relates generally to systems and methods for interacting with virtual objects in a haptic virtual reality environment, and more specifically to systems and methods for modifying virtual objects in a haptic virtual reality environment.
Computers have been long used to design objects in a computerized environment. Examples are CAD (computer-aided design) systems that are used in the design of mechanical parts, such as parts used in an assembly process. For example, a designer may use a CAD system to design parts for an automobile, which are then produced and used in an automobile assembly line. Such CAD systems require significant training and understanding in using the computerized design system. CAD systems are typically difficult to use and lack the freedom and expressiveness of traditional noncomputerized drawing, sketching, and model-making procedures.
One noncomputerized approach involves modeling the object from clay using a traditional sculpting approach based on hand tools and hand sculpting techniques to add, remove, and shape the clay.
This and other approaches suffer from traditional limitations such as the time needed to hand design templates, and the difficulty or inability to recover from mistakes or return to a previous stage of design. Objects created using traditional physical modeling methods cannot be directly incorporated or used in modern digital CAD and computer-aided manufacturing (CAM) processes. In addition, foam and clay approaches cannot be combined readily in the same model.
A more modem approach uses a virtual reality technique to model objects in a computerized virtual environment. Virtual Reality (VR) is an artificial environment constructed by a computer which permits the user to interact with that environment as if the user were actually immersed in the environment. Early VR devices permitted the user to see three-dimensional (3-D) depictions of an artificial environment and to move within that environment. The reality of the VR environment is enhanced by the ability of a user to manipulate virtual objects within the virtual environment using hand motions and gestures. A user may use a virtual tool to manipulate and/or modify a computerized model or virtual object in the virtual environment.
Many existing virtual reality techniques do not provide for a realistic feeling of sculpting by the user. The user cannot feel when virtual tools touch or modify the virtual object. Moreover, in some cases, the virtual tool may pass through the virtual object without any impediment, thereby severely degrading the realism of the experience and the user""s ability to finely control the modification.
Thus, there is a need for a 3-D computerized modeling system that overcomes the problems of CAD techniques and traditional noncomputerized modeling techniques. One object of the present invention is to provide a computerized 3-D virtual reality modeling system that provides the ease and expressiveness of traditional hand model-making approaches as well as integration into the digital design and manufacturing process offered by CAD/CAM techniques.
Being able to feel the virtual object allows the user to resolve visual ambiguities, such as a shape that may appear either concave or convex as perceived by an observer. The user may rely on haptic feedback when modifying the object such as scratching a slight groove in the object, which the user then deepens or expands while receiving feedback through the tool on the shape and current depth of the groove. Feedback also allows the designer to monitor and modulate the cutting rate or amount of change induced by the virtual tool. Haptic feedback also helps the user navigate around and on the surface of the object; that is, using the feel of the object to know where the virtual tool is on the object.
One embodiment of the invention provides a method for a virtual tool including multiple points to interact with a virtual object in a computerized modeling virtual environment. In a 3-D sculpt mode, a user uses a virtual tool to interact with and modify a 3-D virtual object to produce a model.
The invention relates to a method for creating or modifying a virtual object in a haptic virtual environment, including the steps of determining a virtual tool having discrete points for use by the user in the haptic virtual environment; selecting a modification mode for the virtual tool; sensing a location of a user in real space; determining locations of the points of the virtual tool relative to a position of the virtual object; calculating an interaction force between the virtual tool and the virtual object based on the locations of the points and the position of the virtual object; producing a new or modified virtual object by modifying the virtual object based on the modification mode, the locations of the points of the virtual tool, and the location of the virtual object; and outputting the modified virtual object.
In another embodiment of the invention, the method includes determining a virtual surface for the virtual object, and determining a position and orientation of the virtual tool by determining the location of the points of the virtual tool relative to the virtual surface. In a further embodiment the method includes determining the virtual surface to be a virtual isosurface. In one embodiment, the virtual object is a volumetric representation.
In one embodiment the method includes selecting a material removal, a material addition, or a material modification mode. In another embodiment, the method includes determining one or more virtual constraints for the movement of the virtual tool. In another embodiment, the method includes exporting the created or modified virtual object to other formats, including other file types and physical media.
In one embodiment, the method includes modifying a volumetric (voxel-based) object, converting the object to an exported surface, such as a polygonal surface, and exporting the exported surface to a storage media, such as a disk. In another embodiment, the method includes exporting the exported surface to a 3-D printer or stereo-lithographic machine. In a further embodiment, the exported surface is a geometry consisting of one or more non-uniform rational b-splines.
In one embodiment, the method includes importing an imported surface, such as a polygonal or other surface representation, converting it to a volumetric object, and modifying the volumetric object.
In one embodiment the method includes importing a surface, converting it to a volumetric object, modifying the object, converting the object to an exported surface, and exporting the exported surface.
The importing and/or exporting of a surface may be done in a haptic virtual environment, or a virtual environment without haptics.
The invention also relates to a system for modifying a virtual object by a user in a haptic virtual environment. The system includes a virtual tool, a haptic interface device, and a modeling application. The virtual tool includes a plurality of discrete points for use by the user in the haptic virtual environment, wherein the user selects a modification mode for the virtual tool. The haptic interface device senses a location of the user in real space. The modeling application determines locations of the points of the tool relative to a location of the virtual object; calculates an interaction force between the virtual tool and the virtual object based on the locations of the points and the location of the virtual object; produces a modified virtual object by modifying the virtual object based on the modification mode; the locations of the points, and the location of the virtual object; and outputs the new or modified virtual object.
In one embodiment, the virtual object includes a virtual surface and the modeling application determines the position and orientation of the virtual tool by determining the locations of the points relative to the virtual surface. In another embodiment, the virtual surface is a virtual isosurface. In a further embodiment, the virtual object is a volumetric representation.
In another embodiment, the modification mode is a material removal, material addition, or a material modification mode. In a further embodiment, the user of the system determines one or more virtual constraints for the movement of the virtual tool. In an additional embodiment, the modeling application exports the created or modified virtual object to other formats, including other file types and physical media.
In one embodiment, the modeling application modifies a volumetric (voxel-based) object, converts the object to an exported surface, such as a polygonal surface, and exports the exported surface to a storage media, such as a disk. In another embodiment, the modeling application exports the exported surface to a 3-D printer or stereo-lithographic machine. In a further embodiment, the exported surface is a geometry consisting of one or more non-uniform rational b-splines.
In one embodiment, the modeling application imports an imported surface, such as a polygonal or other surface representation, converts it to a volumetric object, and modifies the volumetric object.
In one embodiment the modeling application imports a surface, converts it to a volumetric object, modifies the object, converts the object to an exported surface, and exports the exported surface.
The importing and/or exporting of a surface may be done in a haptic virtual environment, or a virtual environment without haptics.
In another embodiment, the invention relates to a method for interfacing with a virtual object in a haptic virtual environment, including generating a virtual object including a virtual surface in the haptic virtual environment; setting a constraint geometry in the haptic virtual environment; determining a virtual tool for use by the user in the haptic virtual environment; sensing a location of a user in real space; determining a haptic interface location in response to the position of the user in real space; determining a position of the virtual tool in the haptic virtual environment in comparison to the haptic interface location, the virtual surface and the geometric constraints; constraining an action of the virtual tool based on (i) the constraint geometry, (ii) the virtual surface, (iii) the position of the virtual tool, and (iv) the haptic interface location.
In another embodiment, the method includes setting a constraint point, a constraint curve, and a constraint surface. In a further embodiment, the method includes moving the position of the origin of the virtual tool to coincide with the haptic interface location. In one embodiment, the method includes creating or modifying the virtual object based on the position of the virtual tool. In another embodiment, the method includes calculating an interaction force among the constraint geometry, the virtual object, and the virtual tool in response to determining the position of the virtual tool.
In one embodiment, the method includes selecting a modification mode for the virtual tool and modifying the virtual object in response to the modification mode and the position of the virtual tool.
In another embodiment, the method includes constraining the translation of the virtual tool. In a further embodiment, the method includes constraining the rotation of the virtual tool.
The invention also relates to a system for interfacing with a virtual object in a haptic virtual environment, the system including the virtual object having a virtual surface; a virtual tool for use by the user in the haptic virtual environment; a constraint geometry limiting the movement of the virtual tool; a haptic interface device, which senses a location of the user in real space; and a modeling application. The modeling application determines a haptic interface location in the haptic virtual environment in response to the location of the user in real space; determines a position of the virtual tool in comparison to the haptic interface location, the location of the virtual surface and the constraint geometries; constraining an action of the virtual tool based on (i) the constraint geometry, (ii) the virtual surface, (iii) the position of the virtual tool, and (iv) the haptic interface location.
In one embodiment, the constraint geometry is one or more of a constraint point, a constraint curve, and a constraint surface. In another embodiment, the modeling application determines the position of the virtual tool by moving the position of the virtual tool towards the haptic interface location. In a further embodiment, the modeling application modifies the virtual object based on the position of the virtual tool. In another embodiment, the modeling application calculates an interaction force among the constraint geometry, the virtual object, and the virtual tool in response to determining the position of the virtual tool.
In one embodiment, the system includes a modification mode, and the modeling application modifies the virtual object in response to the modification mode and the position of the virtual tool. In another embodiment, the action of the virtual tool includes a translation of the virtual tool. In a further embodiment, the action of the virtual tool includes a rotation of the virtual tool.